Preamble... This article was written back in the summer but, for reasons beyond the author’s control, was a little delayed...
One day, on a hot summer evening, after another entry into the console browser commands like :set content.proxy socks://localhost:9050
, The author of this opus realized that it is impossible to live like this any longer and it’s time to bring access to all sorts of hidden networks, and at the same time bypassing blocking of the name of a well-known organization, to some single “common denominator” for any software in general and browser in particular. How to bring it? Of course, so that the proxy server itself “understands” through which higher proxy to send and receive traffic depending on the entered address. The second goal, which follows from the first, is that upstream proxies can operate as either http or socks and both protocols must be supported by the ingress proxy. Well, the software itself should be more or less up-to-date, so that in case of errors or “wanting features”, you don’t have to sadly look at a lonely turnip on Github, or even on some Sorsforge.
So the goals are set!
The agony of choice
In fact, there was no particular pain. Because, by and large, of the available known proxy servers, two met the requirements. This privoxy
And tinyproxy
. But tinyproxy
turned out to be more lively, more lightweight and simpler, so it was chosen and immediately installed (using the current version of Manjaro Linux as an example).
sudo pacman -Syu tinyproxy
Of course, before this, the packages were already installed tor
And i2pd
. (sudo pacman -Syu tor i2pd
).
Basic tinyproxy setup
So, let’s set up basic redirections so that the regular web goes directly, and *.i2p
And *.onion
through the corresponding parent proxies.
/etc/tinyproxy/tinyproxy.conf:
User tinyproxy
Group tinyproxy
PidFile "/var/run/tinyproxy/tinyproxy.pid"
Port 8888
Listen 127.0.0.1
Timeout 600
DefaultErrorFile "/usr/share/tinyproxy/default.html"
StatFile "/usr/share/tinyproxy/stats.html"
Syslog On
# Set the logging level. Allowed settings are:
# Critical, Error, Warning, Notice, Connect, Info
LogLevel Info
MaxClients 100
MinSpareServers 5
MaxSpareServers 20
StartServers 10
MaxRequestsPerChild 0
Allow 127.0.0.1
ViaProxyName "tinyproxy"
## Parent proxy for TOR hosts
upstream socks5 127.0.0.1:9050 ".onion"
## Parent proxy for I2P hosts
upstream socks5 127.0.0.1:4447 ".i2p"
##### End of static configuration #####
To begin with, almost all parameters in the config remain default.
- Save
- Let's launch:
sudo systemctl enable --now tinyproxy
- Checking:
journalctl -f -u tinyproxy
, We are simultaneously trying to access i2p and onion resources (by setting the browser to use http proxyhttp://localhost:8888
) and see in the log file messages about redirections to the parent proxy's:
июл 20 01:36:16 dell-lnx tinyproxy[198356]: Request (file descriptor 6): GET http://flibusta.i2p/ HTTP/1.1 июл 20 01:36:17 dell-lnx tinyproxy[198356]: Found upstream proxy socks5 127.0.0.1:4447 for flibusta.i2p июл 20 01:36:17 dell-lnx tinyproxy[198356]: opensock: opening connection to 127.0.0.1:4447 июл 20 01:36:17 dell-lnx tinyproxy[198356]: opensock: getaddrinfo returned for 127.0.0.1:4447 июл 20 01:36:17 dell-lnx tinyproxy[198356]: Established connection to socks5 proxy "127.0.0.1" using file descriptor 7. июл 20 01:36:40 dell-lnx tinyproxy[198356]: Closed connection between local client (fd:6) and remote client (fd:7) ... июл 20 01:39:36 dell-lnx tinyproxy[214495]: Found upstream proxy socks5 127.0.0.1:9050 for ilitafrzzgxymv6umx2ux7kbz3imyeko6cnqkvy4nisjjj4qpqkrptid.onion июл 20 01:39:36 dell-lnx tinyproxy[214495]: opensock: opening connection to 127.0.0.1:9050 июл 20 01:39:36 dell-lnx tinyproxy[214495]: opensock: getaddrinfo returned for 127.0.0.1:9050 июл 20 01:39:36 dell-lnx tinyproxy[214495]: Established connection to socks5 proxy "127.0.0.1" using file descriptor 7.
List of "banned.info"»
Well, the proxy combination works basically, now the fun begins - bypassing Roskomnadzor blocking. Unfortunately, tinyproxy
does not support external files with a parent proxy list, but this should not be a barrier. After all, we can generate a monolithic config “on the fly”».
Copy the existing tinyproxy config under a new name:
cp /etc/tinyproxy/tinyproxy.conf /etc/tinyproxy/tinyproxy.conf.static
We slightly edit the new one
/etc/tinyproxy/tinyproxy.conf.static
:LogLevel Info
→LogLevel Error
Create a unit that will clone the project repository ban.info —
sudo systemctl edit --full --force z-i-prepare.service
:
# /etc/systemd/system/z-i-prepare.service [Unit] Description=Zapret Info repository cloner ConditionPathExists=|!/usr/local/lib/z-i/ ConditionFileNotEmpty=|!/usr/local/lib/z-i/dump.csv Wants=local-fs.target After=local-fs.target Wants=network.target After=network.target # [Service] Type=oneshot User=tinyproxy Group=tinyproxy ExecStartPre=+/usr/bin/mkdir -p /usr/local/lib/z-i ExecStartPre=+/usr/bin/chown tinyproxy:tinyproxy /usr/local/lib/z-i ExecStartPre=+/usr/bin/chmod 0750 /usr/local/lib/z-i ExecStart=git clone --depth 1 https://github.com/zapret-info/z-i.git /usr/local/lib/z-i
We create a unit that will generate the tinyproxy config at runtime —
sudo systemctl edit --full --force tinyproxy-cfg-generator.service
:
# /etc/systemd/system/tinyproxy-cfg-generator.service [Unit] After=z-i-prepare.service Wants=z-i-prepare.service # [Service] Type=oneshot User=tinyproxy Group=tinyproxy Environment="PATH=/usr/local/bin:/usr/sbin:/usr/bin" ExecStart=tinyproxy-cfg-gen.sh StandardOutput=file:/run/tinyproxy/tinyproxy.conf
… and the script itself
/usr/local/bin/tinyproxy-cfg-gen.sh
to him:
#!/usr/bin/env sh # tinyproxy-cfg-gen.sh -- tinyproxy dynamic config generator to stdout. awk -F';' '{print "upstream socks5 127.0.0.1:9050 \"" $2"\""}' /usr/local/lib/z-i/dump.csv|tr -d '*'|sort|uniq|grep -v '\s\"\"'>/tmp/tinyproxy.conf.dynamic cat /etc/tinyproxy/tinyproxy.conf.static /tmp/tinyproxy.conf.dynamic
Timer and service that will pump out list updates once a day and restart the main service:
sudo systemctl edit --full --force z-i-update-daily.timer
:
# /etc/systemd/system/z-i-update-daily.timer [Unit] Description=Zapret Info daily update After=network.target Wants=network.target # [Timer] OnCalendar=daily AccuracySec=1h Persistent=true # [Install] WantedBy=timers.target
And the service for it
sudo systemctl edit --full --force z-i-update-daily.service
:
# /etc/systemd/system/z-i-update-daily.service [Unit] Description=Zapret Info daily update service After=network.target Wants=network.target # [Service] Type=oneshot User=tinyproxy Group=tinyproxy ExecStartPre=/usr/bin/git -C /usr/local/lib/z-i pull ExecStart=+/usr/bin/systemctl try-restart tinyproxy.service
Finally, the cherry on the cake, we edit tinyproxy.service to suit our needs —
sudo systemctl edit tinyproxy.service
:
# /etc/systemd/system/tinyproxy.service.d/override.conf [Unit] Wants=network.target Wants=z-i-prepare.service After=z-i-prepare.service Wants=tinyproxy-cfg-generator.service After=tinyproxy-cfg-generator.service # [Service] User=tinyproxy Group=tinyproxy ExecStart= ExecStart=/usr/bin/tinyproxy -c /run/tinyproxy/tinyproxy.conf ExecStopPost=+/usr/bin/rm -rf /run/tinyproxy/tinyproxy.conf
And now, with all this disgrace, we will try to take off ©
sudo systemctl enable --now tinyproxy sudo systemctl enable --now z-i-update-daily.timer
How it works?
A thoughtful reader will certainly be interested in, what are these dances with a tambourine for? Well, in conclusion it doesn’t hurt to clarify some points. Let's go straight to the points of the previous section.
- Everything is simple here. We save in a separate file that part of the configuration that should not change automatically.
- A very important parameter that reduces the loading time of the main service from an hour (SIC!) to about a minute and a half (netbook AMD processor from 2009 and HDD at 5400 rpm). Of course, this is not the only way to improve productivity..
- «bootstrap" unit that always runs, but only runs if the directory is missing
/usr/local/lib/z-i/
or the file is empty/usr/local/lib/z-i/dump.csv
(DirectivesCondition*
). Key--depth 1
allows you to clone only the last commit, not the entire 8 GB. - The main generation of the config and another live hack to improve performance. From csv
awk
-Then the field with the domain is cut out and cleared of unnecessary characters. Rows with an empty domain are deleted, thencat
sends the result tostdout
. and already a unit, thanks to the directiveStandardOutput=
writes all output to a config file in/run
ontmpfs
! According to dependencies, it starts after the “bootstrap” unit from the previous paragraph has completed. - Once a day, starting from zero hours, with jitter per hour, we update the repository and regenerate the config, with a service restart. More precisely, we restart the service with regeneration of the config.
- (and 7.) Well, everything is clear here, launching auxiliary units and the main.
This link is already working a week 2,5 month. Glucose bugs do not seem to have been noticed yet. ready-made configs and scripts live on github, PR are welcome!